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This paper reports on a mixed methods study of 111 AP calculus students’ self-reports of their 
graphing calculator use, comfort, and rationale when choosing between mathematical solutions 
produced with and without a graphing calculator. 


Graphing calculators have taken on an important role in learning mathematics, especially at 
the secondary level (Dion et al., 2001). Graphing calculators have been reported to have been 
used by over 80% of U.S. high school mathematics teachers and are currently allowed on more 
than 70% of the U.S. state’s mandated standardized tests (Texas Instruments, 2010; Weiss, 
Banilower, & Smith, 2001). In addition, they are allowed on 100% of college entrance exams 
and AP calculus, statistics, physics, and chemistry exams (College Board, 2010). One way in 
which a graphing calculator is often promoted in the classroom is as a tool for checking or 
verifying work done by hand (e.g., Doerr & Zangor, 2000; Harskamp, Suhre, & van Streun, 
2000; Hennesey, Fung, & Scanlon, 2001; McCulloch, 2005; 2009; Quesada & Maxwell, 1994). 
The fact that students use the tool to check their work is not surprising. However, the literature 
has not considered the benefits and constraints for students when using the tool in this way. In 
this paper we report on a mixed methods study of 111 high school Advanced Placement (AP) 
calculus students’ uses of graphing calculators to check their work and their methods for 
reconciling any differences between solutions found by hand and with the technology. 

Background 
Graphing Calculator as a Checking Tool 

Doerr and Zangor (2000) investigated the ways in which students use graphing calculators to 
support mathematical learning. They found that the graphing calculator was often used as a 
checking tool “when it was used to check conjectures made by students as they engaged with the 
problem investigations” (p. 156). Quesada and Maxwell (1994) used open-ended questions to 
analyze students’ thoughts on the positive and negative aspects of the graphing calculator. They 
found that a majority of students felt that the ability of the graphing calculator to check answers 
was a positive aspect of the tool. From additional interviews with students, the researchers found, 

The statement ‘the ability to check their answers’ was interpreted by some to mean not only 

the capability of confirming graphically or numerically the answers obtained algebraically, 

but also the ability (new for many of them) of thinking graphically about problems before 

trying to solve them algebraically” (Quesada & Maxwell, 1994, p. 213). 

Other researchers have observed the promotion and use of the graphing calculator as a 
checking or verification tool in the classroom (e.g. Berry, Graham, & Smith, 2006; Doerr & 
Zangor, 2000; McCulloch, 2009), and researchers have seen that the use of the tool in this way 
can be important for creating an understanding of graphing (Hennesey, Fung, and Scanlon, 
2001), and for allowing students to support their analytical work with graphs and tables (e.g. 
Waits & Demana, 1994). Researchers have not yet, however, clearly determined why a student 
chooses to use the tool to check (Berry et al., 2006). 

Confidence and the Graphing Calculator 
Many students recognize the effectiveness of the calculator for confirming or checking the 
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reasonableness of answers, but they struggle when answers obtained on the graphing calculator 
do not match their own expectations or work they have done on paper (Kenney, 2008; 
McCulloch, 2005; 2009). This relates to a level of confidence that students have both for their 
own work and for the graphing calculator as an effective tool in problem solving. Many 
researchers have been able to find improvements in assessment scores when students use 
graphing calculators (e.g., Ellington, 2003), and some have suggested that a possible reason for 
such improvements is that students are more comfortable or confident when they have a graphing 
calculator to use (Dunham, 2000). However, it seems that even when students feel that graphing 
calculators are useful, they may often lack confidence in the calculator’s ability to help them in 
problem solving (Graham, Headlam, Honey, Sharp, & Smith, 2003) and may put more trust in 
their own work than in the calculator. In other cases, students’ confidence in their mathematical 
ability contributes to a lack of perceived need for verification with the graphing calculator 
(Mesa, n.d.) 

On the other hand, there is concern that students can become over-confident in the graphing 
calculator, using the device as a “black box” and blindly accepting calculator output (Doerr & 
Zangor, 2000; Forster & Taylor, 2000). Doerr and Zangor (2000) explain that this occurs when 
learners depend on calculators to produce answers without attending to the meaning, purpose, or 
interpretations of the problem situation. They found that this “black box” use results when 
neither the student nor the teacher provides meaningful strategies for calculator use. Similarly, 
Goos, Galbraith, Renshaw, and Geiger (2003) suggest the calculator can take on the role of 
“master” for the user and that students can become overly dependent on the tool when “lack of 
mathematical understanding prevents them from evaluating the accuracy of the output generated 
by the calculator” (p. 78). When used in these ways, the graphing calculator can become a source 
of mathematical authority for the user (Williams, 1993; Wilson & Krapfl, 1994) and be over- 
used to the point that students rely on the calculator with little critical analysis of the results 
(Burrill et al., 2002). 

Research has not yet looked in detail at the level of confidence that students have for their 
own work or for the graphing calculator itself when using the tool in problem solving. In this 
study, we focus on this issue by examining where students place their trust when conflicts arise 
in problem solving with a graphing calculator. In this study, we address the research questions: 
What do students say they do when a solution produced without technology does not match one 
produced with technology? How do they say they reconcile this situation? 


Methods 
Data Sources 
Advanced Placement (AP) Calculus classes were chosen as the focus for this study because 
the curriculum and expectation of calculator use is relatively consistent nationwide as it is set by 
The College Board. To ensure that the population of students was as diverse as possible, this 
study was set in four high schools in the northeastern United States. High School A is located in 
a low-income urban community, High School B serves students from both suburban and rural 
communities, High School C serves students in an affluent suburban community, and High 
School D serves students in a middle class suburban community. All four of these schools 
provide their AP Calculus students with a graphing calculator to use at home and at school. High 
Schools A, B, and D provided their students with a TI-83+, while High School C provides the 
TI-89 (which has Computer Algebra System (CAS) capabilities). For the purposes of this study, 
the term graphing calculator refers to both calculators. 
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All of the students at these four schools (n = 111; 49 female; 62 male) completed a survey 
instrument designed (based on pilot studies) to identify the ways that they typically use a 
graphing calculator, both in and out of the classroom, and their comfort in doing so. In 
particular, the survey provided data on student demographics, mathematical achievement as 
determined by their math teacher, frequency of graphing calculator use and comfort with the 
tool. In addition, an open ended item was included that read: 

Imagine the following situation: You solved a problem on your own and then used your 
graphing calculator to check your solution. The calculator gave you a different solution 
than the one you got when you worked the problem on your own. Which answer do you 
trust? Why? 
All 111 respondents provided answers to this question on the survey, in many cases surprising 
the researchers with the amount of detail included in their responses. The responses to this open- 
ended item are the focus of this paper. 
Data Analysis 

All categorical data from the survey instrument were entered into an Excel file and each 
written response to the open-ended item above was first coded for solution chosen in the 
reconciliation process (i.e. graphing calculator (GC), non-graphing calculator (non-GC), or 
neither). We then examined all possible associations between descriptive student characteristics 
and solution chosen. Since the descriptive data were categorical in nature, a chi-squared test for 
association was used. Associations were examined between solution choice and each of gender, 
teacher, teacher-rated mathematics ability, and student-reported comfort using a graphing 
calculator. Next, we analyzed the written responses to the open-ended item using a thematic 
content analysis process (Coffey & Atkinson, 1996). After coding each response based on the 
solution ultimately chosen in the reconciliation process, we examined students’ written responses 
for emerging themes within each solution group. This resulted in the development of a codebook 
with 12 data-driven codes. The codes that emerged within each group of solution choices (GC, 
non-GC and neither) directly corresponded with each other, regardless of the ultimate solution 
choice, and fell into four larger categories. Our resulting list of these four categories appears in 
Table 1 below. It is important to note that the assignment of codes to student responses were not 
discrete. For example, the response “If I’m not sure, the calculator. If I’m sure, my answer. I 
probably plugged something wrong into the calc,” was coded as both confidence in math ability 
and careless errors. In the next section, we provide further details and interpretations for the 
coding results. 
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Table 1 

Categories for rationale of solution choice 

Category Definition 

Careless errors Student notes that “careless errors” (either 
arithmetic or syntactical) are possibly the cause of 
any discrepancies between GC and non-GC 
solutions. 

Check work Student notes that either the GC or non-GC (or 


both) solution(s) must be checked for small errors 
and, barring any small errors, ultimately accepted. 
Recognition of GC affordances and Student notes either affordances or limitations of 


limitations the GC in their reasons for accepting or rejecting a 
GC solution. 
Confidence in math ability Student notes acceptance or rejection of a GC 


solution is based on confidence (or lack there of) 
in own math ability. 


Given the situation posed in the item, that of having to 
reconcile a GC produced solution with a different non-GC 
produced solution, 60 out of 111 students (54%) wrote that 
they would ultimately choose a GC produced solution, 39 
(35%) said they would choose their own work (non-GC 
produced solution), and 12 (11%) did not make a definitive 
choice between the two (Figure 1). No associations were 
found between the solution choice and gender 
(X’(2,N =111) =2.649, p = 266), teacher 


(X°(6,N =111) =8.231,p = .222), teacher rated 
mathematics ability (X°(4,N =11) =2.603,p = .626), or 
student reported comfort using a graphing calculator (X’(4,N =111) = 4.051,p = 399). The most 
commonly provided explanations for deciding what to trust, regardless of solution choice, were 
students’ concerns about making careless errors (n = 58, 52%) and students’ suggestions that 
they would check their work before choosing which solution they trust (n = 43, 38%). A 
summary of the frequency of codes within each solution category appears in Table 2 below. 

Table 2 

Code frequency in each solution preference category 


Non-graphing 
calculator 
35% 


Figure 1: Student solution choices 


GC solution Non-GC Neither Total 
solution 
Careless errors 36 18 4 58 
Check work 22 14 7 43 
Recognition of GC 13 6 1 20 
affordances and limitations 
Confidence in math ability 5 6 1 12 


Note: Codes are not discrete 


The following sections report a more detailed look at these findings. We have sorted the 
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findings by the three choices made by students — GC solution, non-GC calculator solution, and 
neither — and include at least two examples of students’ statements as they relate to the categories 
of rational identified. 
Trust the Graphing Calculator Solution 
Careless Errors 
As noted above, 54% of the students responded that they would choose a GC produced 
solution over a non-GC produced one. More than half of these students’ reasoned that they 
would choose the GC solution because it is easy to make careless errors when working by hand. 
The following statements by students in this category exemplify their responses: 
= “JT would trust the calculator because it is easy to make a careless mistake in 
computation when without a calculator and the numbers are large.” 
« “The calculator as long as I am entering it correctly because I have a greater margin 
of error than does the calculator.” 
= “T trust the calculator because of human error.” 
Students not only recognized that careless errors by hand were likely, but some (n = 6) also noted 
that they would use the situation to help identify their errors (e.g. “I would trust the calculator 
because everyone makes mistakes, so I would use that proposed answer and work back and see 
my mistake and fix it”). On the other extreme, a few students noted that they trust their calculator 
solutions so much that they would change their written work to match it. For example, one 
student wrote, “Calculator. I am a very confident calculator user. I’d try to change my work to 
match the calculator answer.” 
Check Work 
Not all of the students automatically identified the GC solution as the one they would trust. 
Sixteen students noted that they would first check their work, both written and GC produced, 
before choosing the GC solution. The following responses are examples of this: 
» “T would double check my work and also how I entered the problem into the 
calculator. If they still don’t match, I would trust the calculator’s answer.” 
= “Well I would actually make sure I plugged in everything correctly into the 
calculator. If that was right, then I would doubt my own solution. So I’d trust the 
calculator answer.” 
Although these responses might imply a similar feeling to those in the careless error category, we 
see these students’ willingness to check all work first as demonstrating more than just blind trust 
in the technology. 
Recognition of GC Affordances 
Justification for choosing a GC produced solution was also attributed to a belief in the 
infallibility of the GC by 13 students. This reasoning was evident in responses such as: 
* “Calculator — we can make math errors but the calculator doesn’t.” 
* “Unlike humans, calculators don’t make computational mistakes for no apparent 
reason.” 
* “Calculator. The only possible error made by a calculator occurs when a wrong 
number, equation, etc. is entered. Room for error on the calculator is restricted.” 
We have identified this reasoning under the broader category of limitations and affordances of 
the graphing calculator due to students’ beliefs that one of the affordances of the calculator is 
that it does not make errors. 
Confidence in Math Ability 
Five of the 60 students reported that they trusted the GC due to a lack of confidence in their 
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own mathematics abilities. For example, students answered: 
* “Calculator, I have no confidence in my math abilities. 
= “JT would trust the calculator because I am usually wrong and I have had success in the 
past with trusting the calculator.” 
« “The calculator. It is better at algebra than me.” 


Interestingly, only one of the students in this category was rated as a relatively lower ability 
student by the teacher, one other was rated average, and the other three were considered to be 
among the strong students according to the teacher. Thus, student responses show a disconnect 
between their confidence in their ability to do mathematics and their actual performance in class. 
Trust the Non-Graphing Calculator Solution 
Careless Errors 
Thirty-five percent of the calculus students reported that they would choose a non-GC 
produced solution in the situation that it did not match a GC produced one. Like those that chose 
GC produced solutions, many in the non-GC category also considered the possibility of careless 
errors when making their decisions (n=17). The difference, however, was that these students 
were not as concerned about errors in their written work as they were about errors they may 
make pressing the calculator buttons. For example, students suggested: 
= “TJ probably entered it incorrectly on the calculator. I’d try to punch it in again.” 
=» “My own answer is probably right; I have finger-calculator problems. Usually the 
reason for the disparity between my value and the calculator value is the lack of 
parentheses or wrong decimal place.” 
Unlike the GC choosers, none of these non-GC students who used the careless error reasoning 
suggested using the situation to help identify their errors. They were not as concerned with 
determining how to get the correct solution on the calculator if they trusted the work they had 
done by hand. 
Check Work 
Thirteen of the non-GC choosers responded that they would trust their non-GC produced 
solution after checking their work for mistakes. For example, students stated: 
*» “Tl check my work again and if I didn’t do anything wrong then Ill trust my work.” 
* “T would double check my work, and then use my answer because I could have easily 
input something wrong in the calculator.” 
As with the GC group, these students were not willing to blindly accept one answer over the 
other, but did demonstrate more confidence in their own work. Many of the reasons in both this 
and the careless error category were related primarily to students’ personal experiences, as 
evidenced by their discussions of what they “often,” “easily,” or “usually” do. 
Recognition of GC Affordances 
Like the GC choosers, non-GC solution choosers noted confidence in their mathematical 
abilities or their beliefs about the affordances or constraints of the GC in their justifications. 
However, unlike the GC produced solution choosers, these students placed the authority in the 
situation with themselves. For example, six responses for the non-GC choosers focused on the 
limitations of the GC as a reason for not trusting its solution. Students shared: 
= “T trust the answer I would have gotten on my own. The calculator does not show all 
the steps and it is easy to make mistakes when putting information into the 
calculator.” 
= “T trust my own, sometimes the graphing calculator comes up with weird answers 
using trig functions or does not find the right answer.” 
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= “T would trust myself because the calculator tends to make mistakes sometimes in 
graphing when the equation isn’t entered properly.” 
It is interesting to note in the last two examples that students place the blame for mistakes on the 
calculator, when in fact mistakes may be a result of their own careless errors. We placed them in 
this category, however, based on the students perceptions of the tool, not our own. 
Confidence in Math Ability 
Six students noted their confidence in their mathematical abilities (rather than their lack of 
confidence) as their reasons for trusting the non-GC solution. Responses included: 
» “As long as I am confident with the answer I got, and was not very unsure with it in 
the first place, then I would trust my own answer not the calculator’s.” 
=» “Myself, because I am able to do the problem on my own, I wouldn’t even need to 
use the calculator.” 
It is important to note that none of the students who noted confidence in their mathematical 
abilities as justification for choosing a non-GC produced solution were considered by their 
teacher to be among the highest ability students as compared to their peers. 
Trust neither solution 
Twelve of the 111 students that participated in the study did not make a definitive choice 
between either the GC or non-GC produced solutions. The responses of these students indicate 
that they do not necessarily value or rely on one solution over the other. Instead, they all noted 
the importance of rechecking their work, on both the GC and paper, to identify errors and to 
understand why the solutions differed. For example, 
* “T recheck the calculator first and then my own answer. I check both and trust 
neither.” 
= “JT don’t trust either until I figure out where the mistake is. I either made a mistake 
with the math or I typed something into the calculator wrong. When the answers 
match, I trust the answer.” 
Again, these students appear to be considering the same things as the GC and non-GC choosers 
(i.e. checking their work and the possibility of careless errors on paper and in button pushing), 
however they place equal value with each method. Most importantly, they do not exhibit blind 
trust in either solution method. However, the responses do suggest that, as long as the two 
solutions do match, students would feel confident with the answer. 


Discussion 

Our findings in this study show that, whether they would choose the GC or non-GC solution 
when a conflict in answers arises, students are considering similar factors: 

"The possibility of careless errors 

= The importance of checking their work 

« The constraints and/or affordances of the graphing calculator 

* Confidence in their mathematical abilities 

More than half of the students chose the GC solution over the non-GC produced solution. 
Many of the students’ reasons for doing so were attributed to lack of confidence in their 
mathematical ability or to an over confidence in the infallibility of the GC. This finding is 
consistent with Goos et al. (2003) who found that students sometimes develop relationships with 
graphing calculators in which the graphing calculator is viewed as the “master”. This suggests 
that it is truly important for teachers to be aware of the issue of mathematical authority. The 
students surveyed here are high school calculus students — often the best and brightest at their 
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schools — and we find that more than half of them are handing the authority in a mathematical 
situation to the tool over themselves. This raises concerns for the results we might find with a 
group of less strong students. 

On the other hand, the large number of students that noted they would choose the GC 
because they are concerned about making careless errors in their written work gives credence to 
the literature that has suggested that students attitudes towards mathematics increases when GC’s 
are available because having the tool increases their levels of confidence (e.g. Dunham, 2000). If 
having a GC available can decrease students’ concerns about the occurrence of careless errors, it 
is possible that students may spend less time worrying about small mistakes and more time 
focusing on thinking deeply about the mathematics. 

It is promising to see that more than half of the students, regardless of solution choice, noted 
the need to check their work. This suggests that these students are thinking critically about what 
might have caused the difference in answers, rather than just accepting one as true. On the other 
hand, it is possible that those students who did not mention checking might have assumed that 
the situation was placed in a testing situation, one in which they may have assumed they would 
not have time to go back and check their work and would thus be forced to choose a solution and 
move on. Such interpretations should be examined more carefully in future studies. 

In conclusion, given the prevalence of graphing calculator use in U.S. high school classrooms 
and standardized tests, these tools are likely going to be a mainstay in school mathematics for 
quite some time (Weiss, Banilower, & Smith, 2001; Texas Instruments, 2010; College Board, 
2010). The results of this study suggest that while such promotion may be beneficial, it needs to 
be handled carefully so that students do not blindly place mathematical authority with the tool 
when reconciling differing solutions. To build a better understanding of these phenomena, 
further research is needed on how graphing calculator use and decision making is being 
promoted by teachers as well as how students perceive this promotion. In addition, it is 
important to investigate if these results hold true for other technology tools such as computer 
algebra systems, spreadsheets, and dynamic geometry systems. 
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